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1.0  Task  Objectives 

The  GigE  Advanced  Imaging  Sensor  program  addressed  communication  to  advanced  imaging 
sensors. 

Key  objectives  of  the  program  included; 

1 .  Incorporate  latest  industry  standard  digital  communication  technology  (gigabit  Ethernet) 
into  existing  advanced  imaging  sensor. 

2.  Design,  fabricate,  test  and  integrate  advanced  sensor  hardware  and  related  software 
drivers. 

3.  Explore  and  determine  unit  and  system  level  design  limits  of  this  communication 
technology  as  applicable  to  advanced  imaging,  situation  awareness  and  survivability 
systems. 

2.0  Technical  Problems 

The  Wide-Eield  Dynamic  Range  (WEDR)  camera  is  a  high  sensitivity,  wide  field-of-regard 
sensor  of  importance  to  future  military  surveillance  and  survivability  systems.  It  is  intended  to 
be  used  in  harsh,  noisy  military  platforms  with  multiple  camera  streams  simultaneously  gathered. 
BAE  Systems  will  be  the  first  to  bring  reliable  high  speed  video  transport  into  the  military 
environment.  By  developing  a  gigabit  Ethernet  (GigE)  video  transport  solution  for  multiple 
cameras  with  low  latency  and  high  data  reliability,  a  flexible  processing  architecture  will  be 
developed  that  allows  optimization  of  processor  throughput  for  real  time  imaging  in  defense 
applications. 

The  following  innovative  features  of  this  concept  include; 

•  Digital  Data  Distribution  —  The  goal  was  a  200x  reduction  in  data  noise,  insulating  the 
data  from  interference  in  electrically-noisy  military  platforms.  Prior  military  systems 
make  use  of  analog  point-to-point  designs;  a  nominal  8-bit  data  acquisition  of  analog 
transmissions  at  best  had  produced  6  effective  bits  of  analog  signal.  Using  direct  digital 
distribution  is  able  to  achieve  at  least  14  effective  bits  effective  accuracy  with  a  16  bit 
resolution. 

•  Multiplexed  Control  and  Synchronization  —  Multiple  video  data  streams  can  be 
sequenced,  allowing  a  single  board  computer  to  gather  data  from  multiple  sensors  as  the 
host  is  ready  for  data,  while  the  sensors  operate  independently  in  parallel.  The  sensors 
provide  metadata  in  the  digital  stream,  allowing  knowledge  of  precise  timing  of  frame 
acquisition.  This  allows  images  to  be  synchronized  with  platform  motion. 

•  Embedded  Design  -  Design  for  use  in  real  time  systems  that  meet  deterministic  schedules 
for  host  applications.  Current  analog-based  systems  have  to  estimate  latency  for  when  a 
commanded  change  to  a  camera  setting  has  actually  taken  place.  An  analog  frame 
grabber  runs  open-loop  and  continuously  grabs  frames  regardless  of  their  generation.  A 
digital  network  allows  the  host  to  initiate  exposures  on  command,  as  well  as  to  know, 
with  certainty,  that  a  commanded  change  is  in  effect  and  data  is  available. 

•  Military  Ruggedization  —  Operation  of  surveillance  video  in  harsh  military  environments, 
to  include  temperature  extremes.  The  thermal  design  of  the  sensor  required  considerable 
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redesign  from  the  COTS  camera.  Additional  a  reliable  iris  drive,  with  precise  index 
sensing,  allows  the  camera  to  be  used  under  both  day  and  night  conditions. 


The  camera  itself  was  essentially  redesigned  to  develop  custom  electrical,  mechanical,  and 
thermal  solutions  to  the  new  interface.  There  are  only  5  boards  at  the  front  now,  CCD,  ADC, 
FPGA,  USB,  and  CLINK  with  the  power  converter,  power  regulator  and  GigE  boards  at  the  rear 
of  the  camera,  heat  sunk  to  the  case  connected  to  the  main  stack  via  a  self  aligning  connector. 
(See  Figures  1  and  2.) 

3.0  General  Methodology 

For  this  program,  BAE  Systems  worked  with  a  commercial  partner,  Phoenix  Engineering,  Inc.  to 
develop  a  reliable  high-speed  video  transport  camera,  suitable  for  the  military  environment.  The 
foundation  for  this  camera  was  an  existing  commercial  camera  built  by  Raptor  Photonics,  the 
EM247.  The  camera  sensor  is  a  low-light,  monochrome  electron-multiplication  charged-coupled 
device  (EMCCD) 

The  camera  components  were  revised  to  meet  military  environmental  and  packaging 
requirements,  and  communications  interfaces  were  updated  in  order  to  be  integrated  with  a 
Commercial  Off-the  Shelf  (COTS)  Single  Board  Computer  (SBC)  and  demonstrated  in  a  digital 
system.  Its  performance  was  evaluated  in  a  thermal  environment  determined  by  realistic  military 
applications. 

By  developing  a  GigE  video  transport  solution  for  multiple  cameras  with  low  latency  and  high 
data  reliability,  a  flexible  processing  architecture  has  been  developed  that  allows  optimization  of 
processor  throughput  for  real  time  imaging  in  defense  applications. 

Previously,  high  speed  video  data  was  transported  to  computers  via  frame  grabbers  (analog  RS- 
170),  USB  2.0,  IEEE  1394  (Firewire)  or  Camera  Eink®.  Analog  transport  is  susceptible  to  a 
noisy  environment,  USB  2.0  and  Eirewire  have  a  5m  distance  constraint,  and  Camera  Link® 
requires  specialized  cabling  and  has  a  distance  constraint  of  10m.  Remote  cameras  on  military 
aircraft  are  often  subject  to  harsh  environments  and  are  located  more  than  10m  from  the  CPU. 

The  general  use  of  GigE  to  transport  video  data  to  a  processor  would  normally  be  subject  to 
unpredictable  latencies  at  both  the  camera  end,  where  the  video  is  converted  to  IP  and  queued  for 
GigE  transport  over  the  GigE  link,  and  at  the  PC,  where  data  are  received  and  transferred  to 
memory.  To  prevent  these  latencies,  BAE  Systems,  in  conjunction  with  Phoenix  Engineering, 
Inc.,  chose  to  use  Pleora  Technologies  iPORT^'^  IP  Engine  and  iPORT  Hydra  software  to  reduce 
these  latencies  and  processing  time.  The  highly  efficient  engine,  which  does  not  use  an 
operating  system,  acquires  image  data,  converts  it  to  IP  packets,  queues  it  for  transfer,  and  sends 
it  over  the  GigE  link. 

The  Automated  Imaging  Association  (AIA)  has  developed  a  GigE  protocol  for  vision  systems 
called  GigE  Vision  (GEV)  to  provide  an  open  framework  for  image  transfer  and  provide 
flexibility  to  system  designers  to  use  less  custom  and  more  available  devices.  The  protocol 
includes  standardization  for  device  identification,  control  protocol,  stream  protocol  and  feature 
mapping  file  (high  level  to  low  level  registers).  BAE  Systems  adhered  to  the  GEV  protocol 
during  the  development  of  the  GigE  Advanced  Imaging  Sensor. 
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BAE  Systems’  program  goal  was  to  collect  real-time  video  data  from  3  cameras  over  GigE  at  a 
30Hz  frame  rate  in  a  VxWorks  environment.  Currently,  our  firmware  is  producing  data  at  up  to 
approximately  35  Hz.  This  capability  enables  inclusion  of  this  high  speed  digital  imaging 
architecture  into  embedded  processing  applications  and/or  data  recording  systems. 


BAE  Systems  approach  was  to  take  an  existing  Wide  Field  of  View  and  Dynamic  Range 
(WFDR)  camera  design  that  currently  outputs  video  via  analog  RS-170  and  digital  USB  2.0,  and 
alter  the  design  to  output  video  via  GigE.  Three  cameras  were  built  and  interface  to  a  standard 
PC  over  Ethernet  and  to  a  GMS  P701  Embedded  System  Controller 


The  goal  of  the  integrated  system  will  be  to  show  complete  end-to-end  performance  of  a  3 
camera  GigE  Vision  based  system,  communicating  and  streaming  30Hz  frame-rate  imagery  to  a 
ruggedized  system  controller  containing  a  customized  GMS  S702  processor,  and  storing  the 
received  imagery  to  an  internal  hard  drive.  Latency  and  processor  loading  will  be  measured,  and 
the  recorded  imagery  will  be  evaluated  against  its  known  input  to  assess  fidelity 

4.0  Technical  Results 

The  Table  below  compares  the  proposed  design  with  the  achieved  results. 


Table  1  -  Camera  Specifications 


Specification 

Current  Design 

Proposed  Design 

Tested 

Sensor 

Optical 

various 

Same 

Electron  Gain 

0  -  15x 

N/A 

Charge  Multiplication. 
Gain 

I  -  1000 

Same 

1  -  1000 

Pixel  Size 

10  urn 

Same 

10  urn 

Wavelength 

400  -  900nm 

Same 

400  -  900nm 

#  of  pixels 

680  X  485  (analog) 

662  X  496  (digital) 

(658x496  active) 

662  X  496 
(658x496  active) 

664x497 
(658x496  active) 

TEC 

Programmable  (-20  to  +30C) 

Same 

-20  to  +30 

Shutter  Control 

1/30*  sec  to  1/2000*  sec 

N/A 

Frame  Integration 

1/30*  sec  to  1  sec  (30  frames) 

N/A 

Sensor  Exposure  Time 

500psec  to  1  sec (lOOpsec 
resolution) 

Same 

500psec  to  1  sec 

Video  I/F 

Analog 

RS-170,Differential,  interlaced 
scan,  NTSC 

N/A 

Digital 

USB,  Progressive  Scan 

GigE,  Progressive  Scan 

GigE,  Progressive 
Scan 

Control  Interface 

RS-485,  115200  baud 

Same 

GigE  or  RS-485, 
115200  baud 

Frame  Transfer  Rate 

30  frames  per  sec  analog  NTSC, 

~3  frames  per  second  digital 

30  fps 

35fps 

Power  Consumption 

<17  Watts 

(5w  additional  auxiliary  heaters) 

<21  Watts 

<  1 1  Watts  @  28V 
DC  (<5W 
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Specification 

Current  Design 

Proposed  Design 

Tested 

Sensor 

additional  heater) 

Weight 

<  38  oz. 

<  2  lbs. 

2.56  lbs 

Firmware  Upgrades 

Via  USB  I/F 

Same 

Via  GigE  or  RS485 

Conneetor 

TBD  p/n  22  pins 
D38999/24WC35PN 

TBD  p/n  22  pins 

22-pin  MIE-STD 
38999 

D38999/24JC35PA 

Operating 

Temperature 

-51“Cto+52“C 

TE  Cooler  maintains  Foeal  Plane 
at  OC  from  -52C  to  +42C  external 
environment 

-51“Cto+52“C 

-51“Cto>+60“C 

5W  heater  allows 

TE  Cooler  to 
maintain  -20  °C 

5.0  Important  Findings  and  Conclusions 

The  digital  WFDR  meets  provides  the  desired  high-resolution,  high-speed  imaging  eapability. 
Due  to  the  possibility  of  setting  the  camera  to  long  integration  times  and  high  EM  gain  as  well  as 
to  short  integration  times  with  closed  iris,  the  camera  is  capable  of  surveillance  applications  from 
the  darkest  night  to  the  brightest  day. 

The  camera  is  capable  of  rapid  switching  between  different  settings  to  grab  frames.  While  the 
manufacturer’s  requirements  allow  100ms  for  the  camera  to  reach  a  new  operating  condition  up 
changing  exposure  time  or  EM  Gain,  it  appears  that  both  these  transitions  occur  much  faster, 
enabling  the  camera  to  produce  images  at  a  new  gain  state  at  its  maximum  rate  of  over  30  Hz. 

A  few  open  issues  in  the  development  are  still  pending.  Phoenix  Engineering  has  committed  to 
upgrade  the  three  cameras  delivered  under  this  contract  at  no  additional  cost. 

•  Iris  drive.  The  current  iris  drive  functions  over  temperature  down  to  -25  deg  C  (-13  deg 
E),  rather  than  the  required  -5 1  deg  C.  A  new  iris  drive  has  been  designed  for  use  over 
the  full  temperature  scale  and  is  being  built.  These  are  expected  to  be  received  by 
Phoenix  mid-December  2009.  Phoenix  will  retrofit  the  three  cameras  at  no  additional 
cost,  if  desired. 

•  Power  Supply  Elnit  noise.  The  new  PSEl  has  appeared  to  introduce  some  noise  into  the 
ADC.  The  camera  manufacture  has  reduced  this  somewhat,  however  some  noticeable 
noise  still  persists  (primarily  quantitatively,  but  much  less  so  visibly).  This  noise  is  most 
observable  at  high  EM  gains,  where  it  causes  the  lowest  several  (usually  5)  bits  of  the 
image  samples  to  be  set  to  0.  This  causes  image  samples  to  be  quantized  to  the  closest 
multiple  of  32,  which  is  not  visible  at  high  EM  gains.  At  lower  EM  gains,  a  faint  pattern 
can,  at  times,  be  distinguished  in  the  image  under  the  right  conditions.  It  is  believed  that 
these  two  effects  are  the  same,  and  Raptor  Photonics  is  investigating  the  root  cause. 

•  Eirmware  updates.  There  are  a  few  other  firmware  updates  required  to  bring  the  cameras 
into  compliance  with  the  specified  requirements.  These  updates  are  deemed  to  be  of  a 
minor  nature  with  little  or  no  technical  risk. 
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6.0  Significant  Hardware  Development 

To  provide  a  GigE  output,  Phoenix  Engineering,  Inc.  has  modified  the  current  camera  design  to 
include  a  COTS  conversion  module  that  converts  a  Camera  Link®  input  to  a  GigE  output. 
Specifically,  a  slightly  modified  PTIOOOVB  iPORT^'^  device  from  Pleora  Industries  has  been 
used.  The  Camera  Link®  input,  although  not  present  in  the  previous  WEDR,  had  been  used  in 
other  Phoenix  cameras,  and  so  the  design  was  implemented  in  a  re-spin  of  one  of  the  WEDR 
CCAs.  In  addition  to  these  two  boards  and  a  modified  power  board,  the  improvement  of  the 
existing  thermal  heat  plane  has  improved  the  thermal  management  of  the  current  design  and 
accommodated  an  expected  additional  two  watts  generated  by  the  PTIOOOVB. 

The  thermal  design  of  the  sensor  required  significant  creativity  and  analysis  to  bring  the  sensor 
into  compliance  with  military  temperature  ranges.  This  challenge  exceeded  our  subcontractor’s 
level  of  proficiency,  requiring  BAE  Systems  to  step  in  and  provide  assistance.  Several  design 
iterations  were  needed  to  build  confidence  prior  to  build.  This  delayed  the  program  from  its 
original  schedule.  Einal  testing  of  the  sensor  has  shown  that  the  resulting  design  meets  the 
specified  operating  conditions. 

Schematics  showing  the  difference  between  the  old  and  new  camera  internals  are  shown  in 
Eigure  and  2. 


Heat 


Support  Ring 


PSU 

Board 


Figure  1  -  Prior  WFDR  Camera  Design 
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Rear  Support 


Gap  Pads  (X3) 


Pleora  NTx-Mini 


Gig-E  Card 


PSU  &  Filter 
Board 


Heatsink 

Plate 


Figure  2  -  New  WFDR  Camera  Design 
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7.0  Special  Comments 

The  cameras  communicate  using  Gigabit  Ethernet  Vision  (GigE  Vision)  protocol  as  the  standard 
between  the  controller  and  sensors.  GigE  Vision  is  a  standard  for  image  transfer  that  employs  the 
widely  used  Gigabit  Ethernet  connection  (1  Gb/s).  Eeveraging  this  emerging  camera  interface 
standard  has  many  advantages.  Existing  hardware  including  network  interface  cards,  CAT-6 
cable,  and  other  communications  devices  can  be  used  to  connect  our  three  sensors  to  the 
controller.  This  interface  will  also  allow  an  easy  upgrade  to  lOGigE  (10  Gb/s)  when  that 
hardware  becomes  available  in  ruggedized  form  and  our  sensor  progresses  to  lOGigE.  Using  the 
GigE  Vision  protocol  allows  interoperability  of  hardware  and  software  from  different  vendors. 

The  enabling  of  fast  image  transfers  through  use  of  the  GigE  Vision  standard  is  a  feature  that  is  a 
great  improvement  over  the  previous  WEDR.  Another  useful  feature  is  the  predictable  timing  of 
image  transfers. 

Our  prior  system  included  an  RS-485  connection  for  communications  and  an  RS-170  connection 
for  analog  (NTSC)  video.  In  addition  to  the  poor  synchronization  between  messages  commands 
and  the  video  stream,  the  system  incurred  a  transient  penalty  whenever  camera  settings  are 
changed,  as  the  video  needs  to  slew  across  settings. 

With  GigE  Vision,  timing  is  predictable  and  will  allow  the  system  to  function  smoothly.  By 
transferring  images  quickly  and  predictably  the  system  will  greatly  improve  its  performance.  The 
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GigE  Vision  standard  also  allows  for  up  to  100m  cable  lengths,  which  allows  the  sensor  to  be 
distributed  as  required. 

The  main  detraetor  from  using  GigE,  is  that  four  twisted  pair  of  wires  (not  including  the  optional 
RS-485)  is  required  for  eommunieation  whereas  the  analog  WEDR  required  two  twisted  pairs 
(RS-170  and  RS-485).  The  increased  weight  is  a  suitable  tradeoff  for  the  increased  speed  and 
performance  of  the  overall  system. 

8.0  Implications  for  Further  Research 

On  the  controller  end,  the  development  of  the  GigE  Vision  driver  was  a  complex  undertaking. 
The  driver  proeessing  occurs  one  layer  up  from  the  media  layers  (layer  4)  and  operate  direetly  on 
the  paeket  datagrams  received  by  the  SBC's  GigE  network  interfaee  controller  (NIC).  While 
several  NIC  manufacturers  are  developing  GEV  interfaces  for  their  products,  none  of  the  efforts 
to-date  have  intended  for  use  in  an  RTOS  environment. 

The  GEV  interface  to  the  camera  is  functional  and  now  needs  to  undergo  the  rigors  of  thorough 
testing  under  conditions  of  other  desired  applications.  Additionally,  more  rigorous  timing  and 
some  environmental  tests  to  identify  the  boundaries  of  performance  should  be  eondueted,  to 
expand  the  seope  of  applicability  of  these  cameras.  Also,  while  our  eurrent  product  solution  uses 
three  separate  Ethernets  to  control  the  camera,  the  suitability  of  multiplexing  all  devices  onto  one 
Ethernet  should  be  tested,  and  limitations  thereof  determined. 

Einally,  the  root  cause  of  the  PSU  noise,  deseribed  in  Seetion  5.0  should  be  fully  determined, 
isolated,  and  ultimately  removed. 

9.0  Standard  Form  298,  August  1998 

Standard  Eorm  298  is  attached. 
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